Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto-detect VT_MYSQL_ROOT, remove cruft #5488

Merged
merged 5 commits into from
Dec 2, 2019
Merged

Auto-detect VT_MYSQL_ROOT, remove cruft #5488

merged 5 commits into from
Dec 2, 2019

Conversation

morgo
Copy link
Contributor

@morgo morgo commented Nov 28, 2019 •
edited
Loading

The changes are:

  • Push the logic for VT_MYSQL_ROOT autodetection into mysqlctl, and then remove the auto-detection and setting of this variable from Docker images (PATH works fine! It's predictable behavior..).
  • MYSQL_FLAVOR is removed, since it has not been required for some time since auto-detection was implemented based on discovering a local binary, and running --version on it.
  • There are also Docker variables for GOTOP/PYTOP, which don't seem to be used. It is a good opportunity to remove some of the ceremony around bootstrapping.
  • The old docker images around building the website and packaging are removed.
  • I don't believe we require pkg-config or mecurial any more.
  • We are not using azure code pipelines.
  • GOPATH,GOBIN, GO111MODULES - we can just set this in the Makefile, since there is no reason to overwrite it.

Signed-off-by: Morgan Tocker tocker@gmail.com

@morgo
Remove unused cruft
a4c3231 
Signed-off-by: Morgan Tocker <tocker@gmail.com>
@morgo
Copy link
Contributor Author

morgo commented Nov 28, 2019

I am currently running rebuilding docker images and testing locally. I will mark this as ready-for-review after it completes.

@morgo
Copy link
Contributor Author

morgo commented Nov 29, 2019

It passed. Prepared_statement test failed, but that is an existing problem:

make docker_bootstrap && go run test.go -pull=false 
...
2019/11/28 17:06:03 mysql57.backup                          	FLAKY (1/2 failed)
2019/11/28 17:06:03 mysql57.backup_mysqlctld                	FLAKY (1/2 failed)
2019/11/28 17:06:03 mysql57.backup_only                     	PASS
2019/11/28 17:06:03 mysql57.backup_transform                	PASS
2019/11/28 17:06:03 mysql57.backup_transform_mysqlctld      	PASS
2019/11/28 17:06:03 mysql57.binlog                          	PASS
2019/11/28 17:06:03 mysql57.cache_invalidation              	PASS
2019/11/28 17:06:03 mysql57.cell_aliases                    	PASS
2019/11/28 17:06:03 mysql57.cell_no_aliases                 	PASS
2019/11/28 17:06:03 mysql57.check_make_parser               	PASS
2019/11/28 17:06:03 mysql57.client_test                     	PASS
2019/11/28 17:06:03 mysql57.cluster_endtoend                	PASS
2019/11/28 17:06:03 mysql57.custom_sharding                 	PASS
2019/11/28 17:06:03 mysql57.e2e_race                        	PASS
2019/11/28 17:06:03 mysql57.encrypted_replication           	PASS
2019/11/28 17:06:03 mysql57.encrypted_transport             	PASS
2019/11/28 17:06:03 mysql57.endtoend                        	PASS
2019/11/28 17:06:03 mysql57.initial_sharding                	PASS
2019/11/28 17:06:03 mysql57.initial_sharding_bytes          	PASS
2019/11/28 17:06:03 mysql57.initial_sharding_multi          	PASS
2019/11/28 17:06:03 mysql57.java                            	PASS
2019/11/28 17:06:03 mysql57.keyrange                        	PASS
2019/11/28 17:06:03 mysql57.keyspace                        	PASS
2019/11/28 17:06:03 mysql57.legacy_resharding               	PASS
2019/11/28 17:06:03 mysql57.local_example                   	PASS
2019/11/28 17:06:03 mysql57.merge_sharding                  	PASS
2019/11/28 17:06:03 mysql57.merge_sharding_bytes            	PASS
2019/11/28 17:06:03 mysql57.messaging                       	PASS
2019/11/28 17:06:03 mysql57.mysql_server                    	PASS
2019/11/28 17:06:03 mysql57.mysqlctl                        	PASS
2019/11/28 17:06:03 mysql57.prepared_statement              	FAIL (3 tries)
2019/11/28 17:06:03 mysql57.python_client                   	PASS
2019/11/28 17:06:03 mysql57.recovery                        	PASS
2019/11/28 17:06:03 mysql57.reparent                        	PASS
2019/11/28 17:06:03 mysql57.resharding                      	PASS
2019/11/28 17:06:03 mysql57.resharding_bytes                	PASS
2019/11/28 17:06:03 mysql57.resharding_rbr                  	PASS
2019/11/28 17:06:03 mysql57.schema                          	PASS
2019/11/28 17:06:03 mysql57.sharded                         	PASS
2019/11/28 17:06:03 mysql57.sharded_recovery                	PASS
2019/11/28 17:06:03 mysql57.tabletmanager                   	PASS
2019/11/28 17:06:03 mysql57.tabletmanager_consul            	PASS
2019/11/28 17:06:03 mysql57.tabletmanager_etcd2             	PASS
2019/11/28 17:06:03 mysql57.unit                            	FLAKY (1/2 failed)
2019/11/28 17:06:03 mysql57.unit_race                       	PASS
2019/11/28 17:06:03 mysql57.update_stream                   	PASS
2019/11/28 17:06:03 mysql57.update_stream_rbr               	PASS
2019/11/28 17:06:03 mysql57.vertical_split                  	PASS
2019/11/28 17:06:03 mysql57.vertical_split_rbr              	PASS
2019/11/28 17:06:03 mysql57.vschema                         	PASS
2019/11/28 17:06:03 mysql57.vtctld                          	PASS
2019/11/28 17:06:03 mysql57.vtctld_web                      	PASS
2019/11/28 17:06:03 mysql57.vtgate_buffer                   	PASS
2019/11/28 17:06:03 mysql57.vtgate_utils                    	PASS
2019/11/28 17:06:03 mysql57.vtgatev2                        	PASS
2019/11/28 17:06:03 mysql57.vtgatev3                        	PASS
2019/11/28 17:06:03 mysql57.vttest_sample                   	PASS
2019/11/28 17:06:03 mysql57.worker                          	PASS
2019/11/28 17:06:03 mysql57.xb_recovery                     	PASS
2019/11/28 17:06:03 mysql57.xtrabackup                      	PASS
2019/11/28 17:06:03 mysql57.xtrabackup_xbstream             	PASS
2019/11/28 17:06:03 ============================================================
2019/11/28 17:06:03 57 PASSED, 3 FLAKY, 1 FAILED, 0 SKIPPED
2019/11/28 17:06:03 Total time: 2h24m37.8s

@morgo morgo marked this pull request as ready for review November 29, 2019 00:59
@morgo morgo changed the title Remove unused cruft Auto-detect VT_MYSQL_ROOT, remove cruft Nov 29, 2019
@morgo
Merge remote-tracking branch 'upstream/master' into morgo-rm-mysql-fl…
af3dcb4 
…avor

Signed-off-by: Morgan Tocker <tocker@gmail.com>
@morgo
Adjust for master
d1d797d 
Signed-off-by: Morgan Tocker <tocker@gmail.com>
@morgo
Copy link
Contributor Author

morgo commented Nov 29, 2019

@sougou There is a race in the VReplication testsuite, that I can reproduce about 20% of the time:

ok  	vitess.io/vitess/go/vt/vttablet/tabletmanager	0.064s
E1129 03:19:34.913983   17312 binlog_player.go:202] error (expected error) in selecting vreplication settings select pos, stop_pos, max_tps, max_replication_lag, state from _vt.vreplication where id=1
E1129 03:19:34.914299   17312 controller.go:144] stream 1: error (expected error) in selecting vreplication settings select pos, stop_pos, max_tps, max_replication_lag, state from _vt.vreplication where id=1, retrying after 10ms
E1129 03:19:35.240267   17312 engine.go:596] Error waiting for pos: MariaDB/0-1-1084, last pos: MariaDB/0-1-1083: context canceled, wait time: 10.4µs
--- FAIL: TestPlayerCopyTableContinuation (3.86s)
##[error]    vcopier_test.go:517: query:
        "update _vt.vreplication set pos='MySQL56/0f4697bc-1257-11ea-a874-000d3a0d951e:1-160', time_updated=1574997581, transaction_timestamp=1574997579, message='' where id=13", does not match query 0:
        "/update _vt.vreplication set state='Running'"
E1129 03:19:44.356040   17312 controller.go:144] stream 25: Duplicate column name 'val2' (errno 1060) (sqlstate 42S21) during query: alter table t1 add column val2 varchar(128), retrying after 5s
E1129 03:19:51.131790   17312 controller.go:144] stream 35: vstream ended, retrying after 1ms
FAIL
FAIL	vitess.io/vitess/go/vt/vttablet/tabletmanager/vreplication	22.294s

I will leave this test as failed so you can inspect. But if you hit the "Re-run checks" button, I guarantee unit will pass :-)

@morgo morgo mentioned this pull request Nov 29, 2019
Closed
@morgo
Copy link
Contributor Author

morgo commented Nov 29, 2019

It also looks like the unit_race test has been finding races, and they are incorrectly not being reported.

@dkhenry
Copy link
Contributor

dkhenry commented Nov 29, 2019

I think @gedgar still uses the packaging to make RPM files

@morgo morgo requested a review from dkhenry November 30, 2019 00:13
@sougou
Copy link
Contributor

sougou commented Dec 1, 2019

--- FAIL: TestPlayerCopyTableContinuation (3.86s)
##[error] vcopier_test.go:517: query:
"update _vt.vreplication set pos='MySQL56/0f4697bc-1257-11ea-a874-000d3a0d951e:1-160', time_updated=1574997581, transaction_timestamp=1574997579, message='' where id=13", does not match query 0:
"/update _vt.vreplication set state='Running'"

I checked this (and have seen this before). It's hard to avoid. It happens when things become so slow (>1s), that the streamer ends up sending a heartbeat event. We can live with this for now. We can later parameterize the heartbeat interval to a bigger value during tests.

@morgo
Fix unit_race
46a9e65 
Signed-off-by: Morgan Tocker <tocker@gmail.com>
@morgo
Merge remote-tracking branch 'upstream/master' into morgo-rm-mysql-fl…
c26c655 
…avor

Signed-off-by: Morgan Tocker <tocker@gmail.com>
@sougou sougou merged commit 966b679 into vitessio:master Dec 2, 2019
@morgo morgo mentioned this pull request Dec 4, 2019
Merged
@rafael rafael mentioned this pull request Dec 11, 2019
Merged
@morgo morgo mentioned this pull request Dec 12, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dkhenry dkhenry dkhenry approved these changes

@derekperkins derekperkins Awaiting requested review from derekperkins

@sougou sougou Awaiting requested review from sougou

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@morgo @dkhenry @sougou